1. Field of the Invention
The present invention relates to a method and an apparatus for displaying real space and virtual space images.
2. Description of Prior Art
Virtual reality (VR) technique is developed to present information prepared by a computer to a man and makes him or her recognize as if he or she is in a real world. By using the virtual reality technique, a man comes into a virtual environment and has the same experience as in a real world. Virtual reality is also called as artificial reality.
Virtual reality technology includes a field called as augmented reality (AR). In the augmented reality, a real space mixed with a virtual space prepared by a computer is presented to a man. That is, augmented reality augments the real space by the virtual space.
In the virtual reality technology, a man is surrounded completely in the virtual environment. On the other hand, the augmented reality technology has an object to increase the quality of the real environment by arranging virtual environment appropriately to the real environment. Visual augmented reality presents an image as virtual environment. Visual augmented reality is referred hereinafter to as augmented reality.
Augmented reality technique will be used for systems, for example, for operation by superposing a computerized tomography scan image of a brain of a patient with the real brain, for supporting construction, repair or the like of a machine by adding comments with computer graphics to each components of the real machine, for evaluating how a furniture looks when it is added to an actual room, and for evaluating how a new building affects the environment in full-scale when it is built in the redevelopment of a city.
See-through head mounted display (STHMD) is used usually for presenting real and virtual spaces at the same time in augmented reality technique. STHMD systems includes a system using an optical system and a system using video signals.
FIG. 1 shows a principle of an augmented reality system with a see-through head mounted display using an optical system. An observer 80 can see a real space image through a half-mirror 81 placed in a line of sight before his or her eyes EY. On the other hand, the observer 80 also sees a virtual space image projected by a projector 82 reflected by the half-mirror 81. Thus, the virtual space image overlaps the real space image. A position detector (PD) 83 is set on the head of the observer 80 to detect a viewing point, and a work station (WS) 84 sends signals of the virtual space image to the projector 82 according to the output of the position detector 83.
FIG. 2 shows another augmented reality system with a see-through head mounted display using video signals. Two cameras 93 are arranged at positions conjugate to the left and right eyes EY of an observer 90 and pick up a real space image reflected by a double sided mirror 91. A work station 95 synthesizes the real space image with a virtual space image, and a projector 92 projects the synthesized image to the mirror 91 and present it to the observer 90. A position detector (PD) 83 is also set on the head of the observer 90 to detect a viewing point in this system.
In order to realize augmented reality, it is necessary to present correctly cover relation so that a farther space or object is covered by a space or object nearer to the observer, besides the position adjustment which is a problem already known. However, the above-mentioned prior art augmented reality systems cannot represent cover relation correctly. In the augmented reality system using an optical system, it is determined that real space is transparent behind an object in virtual space. Thus, an observer sees overlapped images of virtual space and real space, and the cover relation cannot be represented at all. In the augmented reality system using video signals, an object in virtual space always covers another object in real world irrespective of the distance. Then, the cover relation cannot be represented correctly as explained below. When an object in virtual space is nearer than another object in real world, cover relation is represented correctly. On the contrary, when an object in real space is nearer than another object in virtual world, the object in real space is covered by the object in virtual space which is farther than the object in real space, or the cover relation is reversed.
Vision psychology reports that if cover relation is not represented correctly, stereoscopic vision is inhibited, or unexpected stereoscopic vision happens because of the overlapping or the inconsistency of the observed image and the cover relation. Therefore, it is very important for augmented reality technology to represent cover relation correctly.